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Abstract 

A new coding scheme for image transmission over noisy channel is proposed. Similar to standard 
image compression, the scheme includes a linear transform followed by successive refinement scalar 
quantization. Unlike conventional schemes, in the proposed system the quantized transform coefficients 
are linearly mapped into channel symbols using systematic linear encoders. This fixed-to-fixed length 
"linear index coding" approach avoids the use of an explicit entropy coding stage (e.g., arithmetic or 
Huffman coding), which is typically fragile to channel post-decoding residual errors. We use linear 
codes over GF(4), which are particularly suited for this application, since they are matched to the dead- 
zone quantizer symbol alphabet and to the QPSK modulation used on the deep-space communication 
channel. We optimize the proposed system where the linear codes are systematic Raptor codes over 
GF(4). The rateless property of Raptor encoders allows to achieve a "continuum" of coding rates, in order 
to accurately match the channel coding rate to the transmission channel capacity and to the quantized 
source entropy rate for each transform subband and refinement level. Comparisons are provided with 
respect to the concatenation of state-of-the-art image coding and channel coding schemes used by Jet 
Propulsion Laboratories (JPL) for the Mars Exploration Rover (MER) Mission. 
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I. Introduction 

In conventional digital image transmission over noisy channels, the source coding and channel 
coding stages are designed and operated separately. Image coding is usually implemented by a 
linear transformation (e.g., DCT, Wavelet), followed by the transform coefficients quantization 
and by entropy coding of the resulting quantization bits. Due to the lack of robustness of standard 
entropy coding schemes, a few bit errors after the channel decoder may dramatically corrupt 
the decoded image. To prevent this catastrophic error propagation, the source is partitioned into 
segments, such that the effect of errors is spatially confined. In order to preserve integrity, which 
is a strict requirement in deep-space scientific missions, the segments affected by errors are re- 
transmitted at the cost of significant delay and power expenditure. Because of the sharp waterfall 
behavior of the Bit-Error Rate (BER) of the powerful channel coding schemes used in deep- 
space communications, slight changes in the transmission channel quality (e.g., SNR fluctuations 
due to atmospheric conditions or antenna misalignment) result in dramatic degradation of the 
post-decoding BER, producing sequences of highly corrupted segments that need retransmission 

In this paper we consider the application of the Joint Source Channel Coding (JSCC) scheme 
developed in fOzgun-Maria-JSCC-08, flU to the specific problem of deep-space image transmis- 
sion. The proposed JSCC scheme consists of a successive refinement (also referred to as "em- 
bedded") quantizer, and a family of linear codes that directly map the sequences of quantization 
symbols generated at each refinement level into channel codewords. This approach is referred 
to as Quantization with Linear Index Coding (QLIC). The linear mapping of the redundant 
quantization symbols into channel-encoded symbols replaces the non-linear entropy coding stage 
of conventional source encoders. QLIC can achieve the same (optimal) entropy compression rate 
of conventional entropy encoders, but it is much better conditioned in terms of residual error 
propagation. Similar to JPEG2000 Q, we apply Discrete Wavelet Transform to the image and 
then quantize the transform coefficients using a dead-zone quantizer. Since an embedded dead- 
zone quantizer divides the quantization cells into at most three regions at every refinement level, 
the quantization indices are naturally represented as non-binary symbols. Differently from our 
previous work in [J2j , flU, here we use nonbinary Raptor codes (notably, over GF(4)) for QLIC. 
We prove an "isomorphism" between the original source-channel coding problem and a "virtual" 
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purely channel coding problem where the source symbols are sent through an appropriate discrete 
symmetric memoryless channel over GF(4) and the channel-coded symbols are sent through the 
AWGN channel with QPSK modulation, which is the standard in deep-space communications. 
This isomorphism allows us to cast the non-standard code optimization in the source-channel 
coding case as a more familiar optimization for the purely channel coding case, which we solve 
by using a modified EXIT chart technique J6]|. 

The three components the proposed JSCC scheme, namely a wavelet transform, a scalar 
embedded quantizer, and a linear encoding stage, are examined in Sections Unl [IV] and |Vj 
respectively. Section UU introduces the notation used throughout the paper and defines the relevant 
system optimization problem for JSCC based on the concatenation of embedded quantization and 
channel coding in general. In Sec. ivjrl we compare the performance of the proposed scheme 
with the state-of-the art image transmission scheme for deep-space communication channel. 
This baseline scheme is based on the separation of source compression and channel coding. Our 
results show that when the channel quality is perfectly known, the highly optimized baseline 
scheme provides slightly higher efficiency. However, as soon as the channel conditions degrade, 
the proposed JSCC scheme offers significant robustness advantages. In particular, it is able to 
handle fluctuations of the channel SNR as large as 1 dB below its nominal value, with visually 
acceptable quality and without requiring retransmissions. 

II. System Setup 

The deep-space transmission channel is represented by the discrete-time complex baseband 
equivalent model 

y t = fi(x t ) + Zt, t = l,2,..., (1) 

where y t G C, x t £ GF(g) is a coded symbol taking on values in a finite field, \i : GF(g) — > X 
is a labeling map of a signal constellation X = {X , . . . , X q _i] with the elements of GF(g) 
and z t ~ £/V(0, N ) is the complex circularly symmetric AWGN. The channel signal-to-noise 
ratio (SNR) is given by E s /N , where E s = - Y^j=o \ ^j\ 2 is the average power of the signal 

'These results appeared previously in (7) as a conference proceeding. This work includes more details about the scheme and 
derivations. 
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constellation. We indicate by Cx(E s /N ) the maximum achievable rate of channel © when the 
input x t is i.i.d. and uniformly distributed over GF(g). 

A source block of length K is denoted by S e R sxK , where S(z, :) = (S(i, 1), . . . , K)) is 
the i-th row of S, with variance of — -^E[||S(z, :)|| 2 ], is referred to as referred to as the i-th source 
component. A (s x K)-to-N source-channel code for source S and channel (QQ) is formed by an 
encoding function Skx = (x\, . . . , xn), and by a decoding function y = (yx, . . . , y N ) (->■ S. 

Letting = 4E[||S(i, :) — S(i, :)|| 2 ] denote the mean-square error for the i-th component, the 
weighted mean-square error (WMSE) is defined by 

1 s 

D=-Y J v i d i , (2) 

i=l 

where {vi} is a set of non-negative weights that depends on the specific application (see Section 
ITTll) . Let n(-) denote the rate-distortion (R-D) function of the i th source component with respect 
to the MSE distortion. Then the R-D function of S with respect to the WMSE distortion is given 
by 

TZ(D) = min - 2^ r »(^»)> subject to - / ~]vjdi = D, (3) 
s i=i s i=i 

where the optimization is with respect to the values dj > for i — 1, . . . , s. For example, for 

parallel Gaussian sources and equal weights = 1 for all i), © yields the well-known "reverse 

waterfiiling" formula (see (HI Theorem 10.3.3]). For a family of successive refinement source 

codes with R-D functions rj(d), i = 1, . . . , s, assumed to be convex, non-increasing and 

identically zero for d > af, the operational R-D function of the source S is also given by ©. 

Therefore, in the following, TZ(D) is used to denote the actual operational R-D function of for 

some specific, possibly suboptimal, successive refinement source code. 

We define the source-channel bandwidth efficiency of the encoder S^xas the ratio b — 

measured in channel uses per source sample. This corresponds to the familiar notion of "bit 

per pixel" in the case where the source symbols are pixels (image coding) and the channel is 

just a storage device for which one channel use corresponds to storing one bit. By analogy, in 

this paper b will expressed in "symbol per pixel" (spp). It is immediate from the definition of 

2 We shall refer to Cx(E s /No) as "channel capacity" even though, for general constellations, the uniform input probability 
may not be capacity achieving. As a matter of fact, for the case of QPSK considered in the rest of the paper the uniform input 
probability does achieve capacity. 
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R-D function that the minimum distortion D achievable at channel capacity Cx{E s /N ) and 
source-channel bandwidth efficiency b is given by D = 7?.~ 1 (6C^(£' s /iV r )). 

III. Subband Coding 

Images are decomposed into a set of source components by a Discrete Wavelet Transform 
(DWT). In this work we make use of the DWT described in JPEG2000 [|51 for lossy compression. 
With W levels of DWT, the transformed image is partitioned into "subbands". A subband 

decomposition example is given in Fig. [T]-A for W = 3. This produces 3W + 1 = 10 subbands, 
which in the figure are indicated by LLO, HL1, LH1, HH1, HL2, LH2, HH2, HL3, LH3, HH3, 
respectively. The subbands have different lengths, all multiples of the LLO subband length. For 
simplicity, we partition the DWT into source components of the same length, all equal to the the 
length of the LLO subband. This yields s = 2 2W source component blocks of length K = K? / 's, 
where /C x /C indicates the size of the original image in pixels. 

Since this DWT is a bi-orthogonal transform, the MSE distortion in the pixel domain is not 
equal to the MSE distortion in the wavelet domain. In our case, for W — 3, the weight of a 
source component block in subband w = {1, . . . , 10} is given by the w-th coefficient of the 
vector [/ 6 , l 5 h, l 5 h, l 4 h 2 , Z 3 /i 3 , / 3 /i 3 , l 2 h 2 , lh, Ih, h 2 ], where, for the particular DWT considered 
(namely, the CDF 9/7 [0 wavelet), we have / = 1.96 and h = 2.08 0. The subband LLO 
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Figure 1. (A): W = 3, partitioning of an image into 10 subbands and 64 source components. (B): Quantization cell indexing 
for a embedded dead-zone quantizer with p = 1, 2, 3. 



consists approximatetely of a decimated version of the original image. In order to obtain better 
compression in the transform domain, a Discrete Cosine Transform (DCT) is applied to subband 
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LLO so that its energy is "packed" into a very few coefficients The resulting few high-energy 
coefficients are separately encoded and transmitted as part of the header. This is highly protected 
by a sufficiently low rate channel code and is not discussed further in this work since it has 
a negligible contribution to the overall coding length. After extracting these few high-energy 
coefficients, all subbands show similar marginal statistics, well-suited for the embedded dead- 
zone quantizer described in the next section. 



The simplest form of quantization defined in JPEG2000 is a uniform scalar quantizer where 
the center cell's width is twice the width of the other cells, at any resolution level. For example, 
Fig. [TJ-B shows such a quantizer with 3 resolution levels. This scheme, referred to as "dead- 
zone" quantizer, is adopted in this work. We indicate the cell partition at every level by symbols 
{0, 1, 2} as shown in FigQ]-B. The scalar quantization function is denoted by Q : R — > {0, 1, 2} p , 
where 2 P+1 — 1 is the number of quantization regions for the highest level of refinement. Let 
U M = Q(S(i, :)) denote the block of ternary quantization indices, formatted as a P x K array. 
The p-th row of uW, denoted by #'(p, :), is referred to as the p-th "symbol-plane", where 
u^(l,:) corresponds to the coarser refinement and u^(P, :) to the finest. A refinement level 
p consists of all symbol planes from 1 to p. The quantization distortion for the i-th source 
component at refinement level p is denoted by Dq{i,p). 

The quantizer output can be considered as a discrete memoryless source, with entropy rate 
= j^H(u.^) (in bits/source symbol). The chain rule of entropy [H yields ifW = ^ p =1 Hp, 
with HjP = ±H (u«(p, :)| u«(l, :),..., u«(p - 1, :)) , p = 1, . . . , P. Then, the set of R-D 
points achievable by the concatenation of the quantizer using 0, 1, . . . , P quantization levels 
and ideal entropy coding is given by 



where, by definition, Dq(i, 0) = of. Using time-sharing, any point in the convex hull of the 
above achievable points is also achievable. Therefore, the operational R-D curve Ti{d) of the 
scalar quantizer is given by the lower convex envelope of the points in ©. 



IV. Embedded Scalar Quantization 




(4) 



3 Notice: quantization levels indicates that the whole source component is reconstructed at its mean value. 
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By construction, r^d) is piecewise linear, convex and decreasing on the domain Dq(i,P) < 
d < of. As such, it is possible to represent r^d) as the pointwise maximum of the family of 
straight lines joining the pairs of R-D points in (5), for consecutive indices p and p + 1. Using 
this observation in ©, the minimum WMSE distortion with capacity Cx(E s /N ) and bandwidth 
efficiency b is the result of the linear program: 



minimize 

s 



- V" Vidi (5) 
a 

i=i 
1 s 

subject to - J^7i < bC x {E s /N ); D Q (i,P) <d t < of, Vz; 7; > a^di + & ij3 , Vz,p, 

S 8=1 

where a itP d+bi !P is the p-th straight line (for appropriate coefficients (aj )P , &j )P ) obtained by linear 
interpolation of the points in ©) forming r,i(d) as said before. 

While © assumes a capacity achieving channel code, in the proposed JSCC scheme the 
refinement levels (symbol planes) of each source component are encoded by actual codes of finite 
block length. Letting rip denote the number of channel encoded symbols for the p th plane of the 



source component, the total channel coding block length is given by iV = Y^i=i 



P =i 



n. 



Consistent with the definition of the Raptor code overhead for channel coding applications J6]|, 
we define the overhead 0® for JSCC such that nf = ^^jft where KH^ /C X (E S /N ) 
is the information theoretic lower bound to the block length, obtained from the source-channel 
coding converse theorem [8, Theorem 8.13.1]. 

In the case of a family of practical codes characterized by their overhead coefficients {9 P }, the 
computation of the achievable R-D function takes on the same form of ©, where the coefficients 
b^p] are obtained from the linear interpolation of the modified R-D points 



9f ) ),D Q (i,p)\, p = 0,...,P. (6) 




(see H for details). For given code families and block lengths, the overhead factors 9$ can be 
experimentally determined, and used in the system optimization. 

To give an idea of the symbol plane entropies resulting from deep- space images, in © we 
give such values for the first source component (subband LLO after DCT) of a test image from 



4 The details of linear interpolation are trivial and are omitted for the sake of brevity. 
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the Mars Exploration Rover, which will be referred to in the following as image MER1: 

= [0.0562, 0.0825, 0.2147, 0.4453, 0.8639, 1.1872, 1.1917, 1.1118] . (7) 

By examining a large library of such images, we observed that the range of values shown in CD) 
are typical for this application. 

V. Channel Coding Optimization 

In this section we discuss the optimization of the linear channel coding stage. For simplicity, 
we focus on a single discrete source u G GF(g) A with entropy H, to be transmitted over the 
AWGN channel © with capacity Cx(E s /Nq). Obviously, the optimization procedure devised 
here can be applied to each source component and quantization layer pair (i,p), by letting 

(i) (i) 

H = Hp and block length n = n p . 

Linear source codes are known to achieve the entropy rate of memoryless sources ifTOl . Several 
works have considered entropy-achieving fixed-to-fixed linear coding for "almost-lossless" data 
compression IfTTTl . Ifi~2ll . Ifi~3ll . Ifi~4l and lfl"5i Linear data compression codes can be directly 
obtained from linear error correcting codes originally designed for additive-noise discrete mem- 
oryless channels. This is due to the following fact |fi~4"l|. Consider a linear fixed length data 
compression code given by a K x n matrix H which maps the source vector u (of length K) 
to the compressed vector c = uH. The optimal maximum a posteriori (MAP) decoder selects 
u to be the most likely source vector satisfying uH = c. Next, consider a discrete additive 
noise channel y = x + u, where the source u acts as the additive noise. Let x be a codeword 
of the a linear code with parity-check matrix H. The MAP decoder in this case computes the 
syndrome c = yH = uH and finds u to be the most likely noise realization satisfying the 
syndrome equation uH = c. Then, it obtains the MAP decoded codeword as x = y — u. It 
is clear that the optimal decoder for the data compression problem is identical to the optimal 
decoder for the channel coding problem. Therefore, the achieved block error rates are identical. 
As a consequence, if H denotes a sequence (for increasing K) of capacity achieving parity-check 
matrices for the discrete additive noise channel y = x + u, then the same sequence of matrices 
achieves the entropy of the source u. In fact, in this case channel capacity and source entropy 
are related by C = log q — H. 

In order to extend the above argument from pure data compression to the transmission of 
compressed data over a noisy channel it is sufficient to concatenate two linear encoding stages, 



o-(l) o-(l) 
ti x , . . . ,M 8 
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c = uH comp for data compression, and x = cG co d for channel coding. Since the concatenation of 
two linear maps is a linear map, by optimizing over all linear maps (not necessarily decomposed 
as the product H comp G co d), it follows that there must exist good linear joint source-channel 
codes. From now on, we shall indicate this single encoding map by x = uH. This can also 
be interpreted as systematic encoding followed by puncturing of the source symbols. Encoding 
with the systematic generator matrix G = [I, H] yields the systematic codeword [u, x = uH] . 
Then, the source symbols u are completely punctured and only x is transmitted. This approach 
is meaningful from an information theoretic viewpoint since, in the limit of large block length, 
any scheme transmitting the (redundant) source symbols directly over the channel is necessarily 
bounded away from capacity. In fact, the source symbols are non-uniformly distributed with 
entropy H < \ogq and therefore do not follow the capacity-achieving distribution. Viewing 
the encoding map as systematic encoding followed by puncturing will be instrumental to the 
proposed use of systematic Raptor codes for this problem, as discussed later on. 

Up to this point we assumed that the noisy channel is also additive over GF(g), and therefore 
it is "matched" to the source alphabet, so that linearity can be defined. However, in the case of 
deep-space transmission, the channel (OQ) is defined over the complex field, and the codeword 
x is mapped onto a sequence of modulation symbols by the labeling map \i. In order to carry 
over the previous arguments to this case we need a "matching condition" between the additive 
group of the source alphabet GF(g) and an isometry group induced on the signal constellation. 
For this purpose, we consider geometrically uniform constellations as defined in IfToll . 

Definition 1: A signal set X is called geometrically uniform if, given any two points X a ,X b G 
X, there exists an isometry w a ,b '■ C — > C that maps X a into X b while leaving X invariant. 

The set of all isometries that leave X invariant forms the symmetry group of X, under the 
operation of mapping composition. A subgroup Q{X) of the symmetry group of minimal size 
able to generate the whole constellation X as the orbit of any of its points is called a generating 
group IfToll . By definition, |(?(X)| = |X| = q. Given an initial point X G X, we have X = 
{w(X Q ) : w G G(X)}. This induces a one-to-one mapping fi : Q(X) — > X referred to as an 
isometric labeling. The isometric labeling ji induces a group structure on X. At this point, the 
sought "matching" condition can be stated as follows: we let X be a geometrically uniform signal 
constellation admitting a generating group Q(X) isomorphic to the additive group of GF(g). 

For example, a g-PSK signal constellation is geometrically uniform, and admits a generating 
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group formed by the set of rotations of multiples of This group is isomorphic to the additive 
group of Z q (integers modulo q). For q prime, the ring Z q coincides with the field GF(q), therefore 
the generating group of the g-PSK constellation is isomorphic to the additive group of GF(g). 
A possible approach for our code design considers q prime and uses g-PSK as constellation. 
In particular, q = 3 is sufficient to represent the dead-zone quantizer symbols and the 3-PSK 
generating group consists of the rotations Q = {I, R, R 2 } where R is a n/3 rotation in C. 

Another example is provided by q = 4. The additive group of GF(4) = {(0, 0), (1, 0), (0, 1), (1, 1)} 
(binary vectors of length 2, with modulo 2 addition) is isomorphic to the additive group of 
GF(2) x GF(2). This group is isomorphic to the isometry group formed by Q = {I, R x , R y , R xy ], 
where I is identity, R x is reflection with respect to the real axis, R y is reflection with respect 
to the imaginary axis, and R xy = R x R y is reflection with respect to the origin. Note that the 
isometric labeling of the 4-PSK constellation by the elements of GF(4) coincides with the well- 
known Gray Mapping, which is routinely used in deep-space communications. 

With the above conditions on X and its isometric labeling fi, we introduce the following 
notation: w x G G(X) denotes the isometry such that w x ([i(0)) = u> x for a sequence 

x G GF(g) n denotes the sequence of isometries w Xt for t — 1, . . . , n; /i(x) indicates the sequence 
of constellation points fi(x t ) for t — 1, . . . , n. 

We wish to translate the non-conventional source-channel coding problem at hand into a 
channel coding problem defined over a particular channel, that we refer to as the associated 
two-block composite channel. We do so in order to reuse known techniques for optimizing the 
linear encoding matrix H for the associated channel coding problem. 

Definition 2: The associated two-block composite channel is a g-ary input channel where the 
input is divided into two blocks, indicated by v and c, of length K and n, respectively. The first 
block is sent through the discrete additive noise channel defined by s = v — u, where operations 
are over GF(g) and where u has the same statistics of the source. The second block is sent 
through the q-ary AWGN channel defined by r = fi(c) + z, where z ~ CjV(0, N 1), as in the 
original AWGN channel ©• 

For the associated two-block composite channel, we consider the systematic encoder [v, c = 
vH]. Then, we have: 

Theorem 1: The source-channel coding scheme with source u, linear encoder x = uH, 
transmission over the noisy channel y = /i(x) + z, and MAP decoding, is equivalent to a channel 
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coding scheme over the associated two-block composite channel with systematic encoding and 
MAP decoding, in the sense that the error region of the source-channel MAP decoder of the 
former is congruent (via an isometric transformation) to the error region of the MAP decoder 
of the latter, for any source vector u and transmitted information vector v. The isometric 
transformation of the two error regions depends, in general, on u and v. 

Proof: Since the two-block composite channel is symmetric by construction, and the sys- 
tematic code [v, c = vH] is linear, it is immediate to show that the MAP decoding error regions 
for different codewords are mutually congruent. Hence without loss of generality, it is sufficient 
to consider v = 0, yielding the all-zero codeword. The MAP decoder for the source-channel 
coding scheme is given by 

u = arg max exp ( -— ||y - n(pd) || 2 ) Pu(u'). (8) 

u':x'=u'H y 1\q J 

The MAP decoder for the two-block composite channel coding scheme is given by 

v = arg max exp f llr — u(c') II 2 | P/yfv' — s). (9) 

to v':c'=v'H V ^0 / 

Using the properties of the geometrically uniform constellation X, the generic term in the 
maximization of ([8j> can be written as 

exp (fe^) iWu<) = exp ( V(x)+_z-,(xOf ) pu(ul) 

= g p(N°) + »-W-«-W" f ))ll'U(rf) 

= exp (Mg)±iMg^^£j p, (u0 . m 
When v = is transmitted, the generic term in the maximization of © becomes 

ex p fMQ) + s-McQ|| 2 U (v , + u) = exp fllM0) + Z -,(y"-u)H)f\ ^ 

\ — N J \ —N J 

= exp ( W0) + z-^-x)||^ p;f(v , /M11) 

where we used the change of variable v' + u = v" and we defined c" = v"H. 
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The error region of © is given by: 

E (u) = {z e C : u|u is generated by the source} (12) 

For the same realization of u, the error region of ® when the all-zero codeword is transmitted 
is given by: 

£o(u) = {z G C" : v 7^ 0| u is the discrete channel noise} . (13) 

By comparing (flOl) and (fTTj) and noticing that the sets of vectors |(x' — x, u') : u' G GF(g)^} 
and |(c" — x, v") : v" G GF(g)^} are identical, we have that if z G S (u) then u>_ x (z) G £(u) 
and, vice versa, if z G £(u) then w^(z) G £o( u )- Since u> x is an isometry of C n , the congruence 
of the error regions £(u) and £o( u ) is established. ■ 

By noticing that the Gaussian distribution is invariant with respect to isometries, the conditional 
probability of error for the joint source-channel coding scheme P(z G £(u)|u) and for the 
associated channel coding scheme P(z G £o(u)|u) are identical, for all realizations of the source 
vector u. Furthermore, we also show in Appendix [A] that a similar equivalence holds for the 
suboptimal Belief Propagation (BP) decoder ifTTl . in the sense that at every iteration of the 
decoder, the set of messages generated by the message-passing BP decoder for the source- 
channel coding scheme can be mapped into the corresponding set of messages generated by 
the message-passing BP decoder for the associated channel coding scheme by a probability- 
preserving mapping lfT8l . It follows that good systematic codes for the two-block composite 
channel (either under MAP decoding or under BP decoding) yield immediately good codes (with 
identical performance) for the source-channel coding problem. Notice that, with no restriction on 
decoding complexity and block length, successful decoding can be achieved with high probability 
if n > KH/Cx(E s /N ), which is also the Shannon limit for the two-block composite channel. 

Focusing on practical coding design with affordable complexity, the proposed coding opti- 
mization strategy consists of choosing a family of good systematic codes under BP decoding 
for the two-block composite channel. Since the source entropy varies from image to image, 
across the source components i (DWT subbands) and symbol planes p, it is necessary to choose 
families of codes spanning a very wide range of coding rates. Systematic Raptor codes are ideal 
candidates for this application since they can produce parity symbols "on demand", and cover 
a continuum of coding rates. In addition, they have excellent performance under BP decoding. 
Non-universality of Raptor codes for general noisy channels is well-known (see [61), and it 
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is established by the fact that the stability condition on the fraction of degree-2 output nodes 
depends on the channel parameter. Following the approach of O, in Appendix HH we extended 
the stability condition to the case of the two-block composite channel and g-ary Raptor codes. It 
turns out that in this case the stability condition is a function of both the symbol plane entropy 
H = Hp and the channel capacity C = Cx(E s /N ). Hence, the Raptor degree distribution must 
be optimized for each pair of (H, C) values. We perform this optimization using "EXIT charts" 
and linear programming, extending [|6|| and lfi~9l to handle the two-block composite channel. 

Before entering the details of the EXIT chart analysis and Raptor code optimization, a final 
remark on the signal constellation is in order. Since the dead-zone quantizer symbols are ternary, 
q must be at least 3. We considered both 3-PSK and QPSK (with Gray Mapping) constellations. 
Although 3-PSK is more naturally matched to the ternary source alphabet, the QPSK modulation 
has higher capacity. Hence, it is not a priori obvious which of the two constellation performs 
better in our context. In our experiments we observed that QLIC with q = 3, using the 3-PSK 
constellation, did not provide any improvement over the case q = 4 with the QPSK constellation. 
Since QPSK with Gray mapping is standardized in deep-space communications, and the BP 
decoder is simplified for powers of 2 field size (see |fi~9l ), q = 4 represents a better and more 
natural choice. Thus, in the following we only focus on QPSK and codes over GF(4). 

A. EXIT Chart Analysis for the Two-Block Composite Channel 

Check nodes 



Variable nodes 
Input nodes 

Output nodes 

Figure 2. The Tanner Graph of a Raptor Code with LDPC code. 

We assume that the reader is familiar with Raptor codes, their systematic encoding and BP 
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iterative decoding, and with the Gaussian approximation EXIT chart analysis technique of BP 
decoding for standard binary codes over memoryless binary-input output- symmetric channels 
(see [6]). Here, we focus on the aspects specific to our problem. 

A Raptor code is formed by the concatenation of a pre-code, here implemented by a high 
rate regular LDPC code, and an "LT" code, which is a low-density generator matrix code with 
a special generator matrix degree distribution J6J. For the Tanner graph of the LT code, we 
define the input nodes and the output nodes. For the Tanner graph of the LDPC code, we define 
the variable nodes and the check nodes (see Fig. 0. We consider Raptor codes over GF(4) 
with systematic encoding. The first K output symbols of the Tanner graph of Fig. [2] are the 
systematic symbols, corresponding to the source block u. The remaining n output nodes are the 
non-systematic (parity) symbols, corresponding to the codeword x. Thanks to the equivalence 
of Theorem Q] and to the analogous equivalence for BP decoding 0, [fT8l . we consider the 
transmission of the Raptor codeword over the two-block composite channel where the first block 
of K symbols go through the additive noise over GF(4) with noise identically distributed as the 
source vector u, and the second block of n symbols is mapped onto QPSK by Gray mapping 
and is sent through the AWGN channel ©. Hence, the rest of this section is dedicated to the 
Raptor code ensemble optimization (namely, the optimization of its degree distribution) for the 
associated two-block composite channel. 

For codes over GF(4) we use the Gaussian approximation approach proposed in lfi~9l . In par- 
ticular, the conditional distribution of each message L in Log-Likelihood Ratio (LLR) domain |f] 
is assumed to be Gaussian L ~ Af(vl, £„), where [E l) ] i . = 2v for i — j and pu]^- = v for 
i ^ j. It can be noticed that the conditional distribution depends only on a single parameter 
v thanks to symmetry and permutation invariance assumption of the messages as defined in 
|fl~9l . Letting V the code variable corresponding to the edge message L, we define the mutual 
information function J(v) = I(V; L) = 1 — E [log 4 (l + Yli=i e ~ Li )] ■ We use base-4 logarithm 
for mutual information calculations, hence in these sections H and C are in units of two bits 
per source symbol or per channel symbol, respectively. 

The EXIT chart is the mapping function of a multidimensional dynamic system that describes 

5 The BP messages for q-ary codes can be either represented as probability vectors (of length q) or as LLR vectors of length 
q — 1. If m is a message in the probability domain, the corresponding message in the LLR domain, denoted by L, has elements 
Li = log(mo/mj) for i = 0, . . . ,q — 1. 
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the evolution of the mutual information between the Tanner graph variables and the corresponding 
messages passed along the Tanner graph edges by the BP decoder. The stationary points of such 
dynamic system are given as the solutions of a set of EXIT chart fixed-point equations, given 
in terms of the following state variables: 

- x denotes the average mutual information between a randomly chosen input node symbol and 
the corresponding message sent downward to an adjacent edge (from input to output nodes). 
See Fig. El 

- y denotes the average mutual information between a randomly chosen input node symbol and 
the corresponding message received upward from an adjacent edge (from output to input nodes). 

- X denotes the average mutual information between a randomly chosen variable node symbol 
and the corresponding message sent upward to an adjacent edge (from variable to check nodes). 

- Y denotes the average mutual information between a randomly chosen variable node symbol 
and the corresponding message received downward from an adjacent edge (from check to variable 
nodes). 

The degree distributions for the Tanner graph in Fig. [2] are defined as follows: 

- For the LDPC code, we let \{x) = ^iKx 1 ' 1 and p(x) = Ylj Pj x ^ 1 denote the generating 
functions of the edge-centric left and right degree distributions, and we let 

A(x) = J2 A * xi = (f Q Hu)*") I Q£ A(«)du^ , 
denote the node-centric left degree distribution. 

- For the LT code, we let u{x) = J2 i liX 1-1 denote the edge-centric degree distribution of the 
input nodes, and we let co(x) = J2j denote the edge-centric degree distribution of the 
"output nodes". The node-centric degree distribution of the output nodes is given by 

Vt(x) = ^^fijX- 7 = ui{u)duj I (^j tu(u)dv^j . 

- For the concatenation of the LT code with the LDPC code we also have the node-centric degree 
distribution of the LT input nodes. This is given by 

2( X ) = ^\X 1 = ^ L{u)du^J I ^ i(u)dv^ . 

Note that for large number of nodes we have the following approximation for 2(x) ~ e a ^ x ~^ = 
J2 n a "n\ " xU wnere a = 1S me average node degree for the input nodes [6J. Hence l(x) 
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is approximated by the following coefficients 

t i = a i - 1 e _a /(z-l)!. (14) 

The capacities of the first and second components of the two-block composite channel are l — H 
and C, respectively. A random edge (o, v) is connected with probability 7 = Kj (K + n) to 
the first block and with probability 1 — 7 to the second block. As a consequence, it is just a 
matter of a simple exercise to obtain the following EXIT equations for the LT code component 
(Detailed derivations for binary Raptor codes can be found in our previous work [0): 

x = ^^A^J((2-l)J- 1 (y) + fcJ- 1 (Y)), (15) 

k i 

y = 1 - 5>,{ 7 J((.7 - 1)^(1 - x) + J-\H)) + (1 - 7) J((j - l)J-\l - x) + J-\l -C))). 

j 

(16) 

Also, notice that 7 = r lt r ldpc , where r h = ^ ^ and r ldpc = 1 - gj^j. are the coding rates 
of the LT code and of the LDPC code, respectively. 

The EXIT equations for the LDPC component are well-known and are given by: 

X = ^^AfcV^-l^m+a-^y)), (17) 

k i 

Y = l-^J^-iy-^l-X)). (18) 

Eventually, (fT5l) . (fl6l) . (TTTI) . and (fT8l) form the system of fixed-point equations describing the 
stationary points the EXIT chart of the concatenated LT - LDPC graph, with parameters H, C 
and 7, and the degree sequences u, 1, p and A. 

The error probability of the output nodes corresponding to the first block of K output nodes, 
sent through the discrete additive noise component of the two-block channel, is identical to the 
error probability of the source symbols in the source-channel equivalent problem. Therefore, 
the key quantity of interest for the performance of the JSCC scheme is the error probability 
of such output nodes. This is can be obtained, within the assumptions of EXIT chart approx- 
imation, as follows. The mean of the LLR of such an output node of degree j is given by 
Vj = J^ 1 (1 — J (jJ~ l (\ — x))) + — H). By the channel symmetry and the code linearity, 
the EXIT chart is derived under all-zero codeword assumption. Hence, decoding is successful 
if the LLR vector has positive components. For an output node of degree j, this results in the 
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symbol error rate (SER) 1 — P(L > 0), with L ~ Af(vjl, S„.). By averaging over the degree 
distribution, the desired average SER is given by 



B. LT Degree Distribution Optimization 

For simplicity, we fix the LDPC code to be a regular (2, 100) code (ri dpc = 0.98). For this 
LDPC code, we find the mutual information threshold 34>(c0 ( usm g (TTVT) and ([781) ) such that for 
y > 34)( a ) the LDPC EXIT converges to Y = 1, with stand-alone iterations. The value of 34) (a) 
depends on the LT input degree distribution t{x), which in turns depends on a via (TT41) . The 
function 34) ( a ) * s monotonically decreasing. Therefore, higher values of a yield less restrictive 
requirements for the mutual information that the LT code must attain in order to allow the LDPC 
code to converge to Y = 1 (vanishing error probability). On the other hand, larger values of a 
yield smaller LT coding rate r\ t , and therefore are more conservative with respect to the system 
bandwidth efficiency. 

Next, we use (fl"5l) and (fl6l) to eliminate x and write y recursively. The fixed-point equation for 
y depends on the input Y coming from the LDPC graph. In order to obtain a tractable problem, 
we decouple the system of equations (I17H18I) and (I15H16I) the target mutual information 34) (oO 
and disregarding the feedback from LDPC to LT in the BP decoder (i.e., letting Y = in 
(fl"5T) . This is equivalent to running BP with the following schedule: first iterate the LT code till 
convergence, and then iterate the LDPC code till convergence. The resulting recursion mapping 
function ff ,C ^' a {y) for a degree- j output node is given by 



ff' c '"' a (y) = {7^^(j-i)^ 1 (i-E^(( i - 1 ) J " 1 (y))) + ^ 1 ( i7 )j 

+(i - 7) j ( (j - \)r\\ - nJ((i - iy -1 (y))) + J~\i - O) ) }(20) 



In order to ensure this condition, we sample the interval [0, 34)( Q 0] on a sufficiently fine grid 
of points {yi}, and obtain a set of linear constraints for the variables {coj}. The code ensemble 




(19) 



We conclude that the LT EXIT recursion converges to the target 34) ( a ) if 




(21) 
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optimization consists of maximizing r lt for given H, C pair, subject to the condition that the 
BP decoder converges to vanishing error probability. The optimization variables are {cuj} and a. 
Since the LDPC code is fixed, 7 is a function of a, {tOj}. In order to linearize the constraints in 
{ujj} we replace 7 in (|2TT) with its ideal value C/(C + H), arguing that good codes must have 
7 « C/(C+ H). This yields the optimization problem: 

mm a mm {uJj} a^ — 
3 3 

s. t. 2^/^ = 1) > 0, 

3 

y t <l-J2"3f 3 H,C ^' a (y*)i V yi e[0,y O (a)]. (22) 



For fixed a, (1221) is a linear program with respect to {w,}. Hence, we can run an educated line 
search with respect to the scalar variable a and, for each tentative a, easily optimize over {ujj}. 
Let ^(v) = E , then the stability condition obtained in Appendix |B] reads 

ma: 

n > ^/ rid P c (21.) 

2 " 2 ( 7 * ( J-i(l _ H )) + (1 - 7)* (J-KC))) ' 

We accept the solution of the optimization if ( 1471 ) is satisfied. Otherwise, the optimization is 

re-run with a more conservative value of a. 



VI. Results 

We present the performance of the QLIC scheme and compare it with the baseline (state-of-the 
art) system used by Jet Propulsion Laboratories (JPL) for the Mars Exploration Rover (MER) 
Mission. For the purpose of this comparison, we briefly present the current baseline system. The 
scheme is based on a separated source and channel coding approach, concatenating an image 
coding scheme called ICER [1J with standard codes for deep- space communications ll20l . [1211 . 

ICER is a successive refinement, wavelet-based image compressor based on the same principles 
of JPEG2000, including image segmentation, DWT, quantization, and entropy coding of the 
blocks of quantization indices using interleaved entropy coding and an adaptive probability esti- 
mator based on context models [1]. These components differ from their JPEG2000 counterparts 
in order to handle specific needs of scientific images for deep-space exploration. ICER makes 
use of a reversible integer-valued DWT |[22| so that, if all the subbands data are fully transmitted, 
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lossless reconstruction can be obtained. Since subband coefficients are integer values, the dead- 
zone quantizer is also modified to work for integer values as described in HI. The quantization 
precision for each subband and the selection of which subbands should be transmitted in order 
to minimize the total number of bits subject to a given target reconstruction Peak SNPv[] (PSNR) 
are established dynamically, based on the actual image to be encoded, according to the relative 
importance of each subband. The resulting priority-ordered bit planes are encoded one by one, 
until the target PSNR (or total bit budget) is reached. ICER and JPEG2000 (using either lossless- 
5/3 integer DWT or lossy-9/7 DWT) provide similar pure image compression performances, i.e., 
when used on noiseless channels flU. In 11231 we found that the pure compression performance 
of the proposed QLIC scheme is also almost identical to ICER. This provides a good sanity 
check for QLIC, which is not inferior to the state-of-the art as far as pure image compression 
is concerned. 

In order to increase robustness against channel errors, ICER partitions the image into segments. 
A segment "loosely" corresponds to a rectangular region of the image (although in practice a 
more sophisticated adaptive segmentation scheme is used). Each image segment is compressed 
independently. In this way, the error propagation introduced by possible residual post-decoding 
channel errors is limited to within a segment. The encoded bits corresponding to the segments 
are concatenated and divided into fixed-length frames, that are separately channel-encoded at 
channel coding rate R c . This takes on values in a finite set of possible coding rates supported 
by the family of deep-space channel coding schemes. The channel coding rate R c is chosen 
according to the channel SNR. The presence of residual errors is detected with probability close 
to 1 using standard error detection techniques, and the frames with residual post-decoding errors 
are erased. Frame erasure is the main cause of data loss in the baseline JPL scheme 0}. 

Since ICER is a progressive image compressor, all successfully decoded frames of a segment 
before the first erased frame can be used for source reconstruction. The reconstruction quality 
of a segment depends on the position of the first frame erasure (of course, the highest quality is 
obtained if no erasure occurs). No unequal error protection is used for the sequence of successive 
frames forming a segment. Therefore, all frames have the same erasure probability HI. As a 

6 The reconstruction PSNR is defined as PSNR = 101og 10 2 ~ x where D is the WMSE distortion and where i = 12, since 
for MER mission each pixel is a 12-bit value in the original image. 
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consequence, segments may be reconstructed at very different quality level, depending on the 
presence and position of frame erasures. If a segment achieves too poor reconstruction quality, 
the retransmission of the whole segment is requested. In MER, retransmissions are possible with 
a delay roughly equal to the round trip time between Earth and Mars which is between 7-30 
minutes (24]. In addition, retransmissions require storing the images on the deep-space probe, 
for a long time and a feedback channel from Earth to Mars for retransmission requests. 

In our comparison, we consider just the spectral efficiency of the "active transmission" phase, 
i.e., as defined by the parameter b. This is the ratio between channel uses (including retrans- 
missions) and source samples. The comparisons reported here do not take into account the long 
idle times and the enormous delay incurred by retransmissions, because the "cost" of these 
system aspects is difficult to quantify from a communication theoretic viewpoint. However, we 
hasten to say that the proposed JSCC scheme does not require retransmissions unless the channel 
SNR dramatically changes with respect to the nominal value assumed at the transmitter. Hence, 
although the spectral efficiency performance is slightly inferior to the baseline system, this built- 
in robustness able to avoid retransmissions is a very attractive feature in terms of delay and 
system simplification. First, we considered a scenario where the target PSNR is fixed. For a 




Figure 3. FER vs Eb/N curves for block length 16Kb. 

given set of test images H we compare the two schemes in terms of b versus E s /N , for the 
same target PSNR. The Frame Erasure Rate (FER) of the baseline system is a function of the 

7 Provided by JPL-MER Mission Group. 
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channel SNR and of the channel coding rate R c used. The number of transmissions necessary 
for the successful reconstruction of a segment is a geometric random variable with success 
probability that depends on the FER, the number of frames F in a segment, and on the target 
PSNR. As mentioned before, the reconstruction quality of a segment depends on the position 
of the first erased frame in the segment. Upper and lower bounds to the success probability 
developed in ll23l show that, for the typically very high target PSNR required by deep-space 
scientific imaging, the success probability is tightly approximated by (1 — FER) F (i.e., a segment 
is retransmitted whenever a frame is in error, irrespectively of its position). For a given channel 
SNR, the baseline scheme chooses the deep-space channel code with maximum rate R c , subject 
to the condition that the FER must be smaller than a target threshold (a typical target is 1CT 6 ). 
The target FER is fixed in order to achieve a desired, and typically very small, retransmission 
probability. For a well matched SNR and rate pair, the FER is effectively very small and the 
expected number of re-transmission is insignificant. In this case, b is very close to the "one-shot" 
transmission value, i.e. B/(2R C ), where B is the number of ICER-encoded bits per pixel at the 
given target PSNR and the factor 2 comes from the fact that QPSK transmits 2 coded bits per 
channel use. 

For each fixed R c , the corresponding b vs. E s /N curve has a very pronounced "L" shape, due 
to the sharp waterfall of the FER (see Fig. [3]). Hence, the SNR axis can be split into intervals, 
where each interval corresponds to the range of E s /N values for which a given coding rate 
"dominates", i.e. yields the best efficiency (including retransmissions). If E s /N is known in 
advance, and the cost of retransmissions is neglected, for each SNR falling in a given interval, 
the corresponding coding rate is selected. For the example considered in this paper, the target 
PSNR is 49 dB and the MER1 image (1024 x 1024 BW uncoded 12-bit per pixel) is used. Fig. 
|4]-A compares the resulting b vs E s /N performance of JSCC and of the baseline scheme. The 
following comments are in order: 

- The (*)-curve corresponds to considering ideal capacity achieving codes for each symbol plane 
in the JSCC scheme. This represents the best possible performance for the DWT and quantization 
scheme used in the proposed system. This curve is also a very good approximation of the 
performance of a separated scheme based on QLIC or ICER for pure compression, concatenated 
with an ideal capacity achieving channel code, since the pure source compression performance 
of ICER and QLIC is essentially indistinguishable 11231 . 
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- The performance of the actual baseline scheme is shown as a superposition of four L-shaped 
curves, each of which corresponding to one of the codes whose FER performance is shown in 
Fig. [3J as explained before. The steep increase of b for small degradation of E s /N beyond the 
"knee" point of each L-shaped curve indicates that if the channel quality degrades slightly below 
the threshold at which each channel code yields small FER, then the number of retransmissions 
per segment increases dramatically. If such channel quality degradation occurs (e.g., atmospheric 
propagation phenomena, rain conditions, antenna alignment fluctuations), then the conventional 
system folds back onto a more conservative channel coding rate, and its performance moves on 
the L-shaped curve to the left. The channel coding rate value R c corresponding to each curve 
is also shown in Fig. |4]A.. 

- Before designing degree distributions specifically for the Raptor codes over GF(4) as described 
in Sec. IV-A[ we first used the degree distribution 

n(x) = 0.008x + 0.494x 2 + 0.166x 3 + 0073x 4 + 0.083x 5 

+0.056x 8 + 0.037x 9 + 0.056x 19 + 0.025x 65 + 0.003a; 66 , (24) 

given in ll25l for binary Raptor codes. The EXIT chart infinite- length performance and finite 
length simulations for this non-optimized non-binary case are shown in Fig. @]-A as the (— .)- 
curve and the (D)-curve, respectively. 

- In Sec. |V]we noted that the proposed method for the Raptor code degree distribution optimiza- 
tion in (1221) is a linear program when the parameter a and the LDPC code are fixed. As seen 
in Sec. IV-Bl r\ t is a function of a and oj(-). Hence, the aim is to maximize r\ t by optimizing 
cj(-) for fixed a, at each value of E s /N . The value of a for given E s /N is obtained using 
the non-optimized code simulations as follows: for the non-optimized ((— .) and (□)) cases the 
distribution u(-) is given by (1241) and r\ t is provided by the simulation at each E s /N point. 
Then, we obtain the corresponding a by using u(-) and r\ t . Finally, for this fixed pair of a and 
E s /N , we use the linear program in order to optimize the LT degree distribution. Although an 
exhaustive search over the feasible range of a may yield further improvements, the above simple 
method already provides a noticeable performance enhancement both in terms of the EXIT chart 
infinite-length performance (o) and in terms of the finite-length simulation (o), with respect to 
the corresponding non-optimized curves (— .) and (□). 

Next, we focus on a particular channel SNR value (in particular, we choose E s /N = 3 dB), 

DRAFT October 3, 2012 



23 



and provide a zoomed version of Fig. SJ-A around this value in Fig. |4]-B. For this SNR, the 
channel code with rate R c = 3/4 yields the best performance for the conventional system. Now 
we consider the case of a mismatch between the actual and the nominal channel quality, i.e., 
we assume that the transmitter chooses the optimal scheme (baseline or proposed JSCC) for 
the nominal E s /N = 3 dB, but the actual value of E s /N is less than 3 dB. In this case, the 
efficiency b of the baseline system significantly decreases due to the retransmissions. At a certain 
point, as the channel conditions worsens, the baseline system switches to the next lower channel 
coding rate R c — 1/2. This happens at E s /N pa 2.8 dB. The proposed JSCC scheme has a better 
built-in robustness to handle mismatched channel conditions, thanks to the QLIC linear map. We 
observe that the JSCC scheme optimized for E s /N = 3 dB and with no retransmission yields 
constant b and a slight degradation of the reconstruction PSNR over the range of channel SNR. 
Due to mismatched channel conditions, there will be some residual error in the symbol planes. 
However, as seen in Fig. [5]-b,-c,-d,-e, the perceptive quality of the reconstructed image (and the 
reconstruction PSNR) gracefully degrades and the perceived image quality is acceptable over a 
wide range of channel SNRs, since there are no artificial "block effects" due to segment losses, 
even though the channel SNR is as far as 1 dB less than its nominal value of 3 dB. Similar 
behaviors have been observed by extensive experimentation, not reported here for the sake of 
brevity and space constraints. 




Figure 4. (A): b vs E s /Nq trade-off curves for various schemes. (B): Focusing on E 3 /No — 3 dB point. The image 
reconstructions at various mismatched SNR values, indicated by (b), (c), (d), (e) in (B), are shown in Fig. [5] 
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(d) (e) 

Figure 5. Original MER1 image (a), and different image reconstructions at E s /Nq — 3 dB, and PSNR = 49 dB (b); at 
E s /N = 2.8 dB, and PSNR = 48.19 dB (c); at E s /N = 2.5 dB, and PSNR = 45.63 dB (d); at E S /N Q = 2 dB and PSNR 
= 38.60 dB (e). Notice that visible artifacts due to residual channel errors can be seen only in figure (e). 

VII. Conclusions 

We proposed a new coding scheme for digital image transmission over a discrete-time AWGN 
channel. The scheme is based on the concatenation of a standard DWT, decomposing the 
image into blocks of subband coefficients, and an embedded dead-zone quantizer that produces 
sequences of ternary quantization indices for the successive refinement "planes" of each subband. 
Then, the redundant symbol planes are mapped linearly into channel codewords, which are 
modulated into constellation symbols and sent over the discrete-time AWGN channel. We showed 
that if the quantization indices symbol alphabet additive group structure is matched to the signal 
constellation generating group structure, the modulation mapping is an isometric labeling, and 
the source-channel encoder is linear, then the source-channel coding problem is equivalent to a 
channel coding problem over a composite two-block channel, where the first block corresponds 
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to the transmission over a discrete additive noise channel with noise statistics identical to the 
source statistics of the original source-channel coding problem, and the second block is the 
AWGN channel with the isometric labeling included as part of the channel. This equivalence 
holds for both the optimal MAP decoder and the suboptimal, low-complexity, BP decoder. This 
allows us to optimize the source-channel coding ensemble as if it was a channel coding ensemble 
for the equivalent channel. In particular, we propose to use Raptor codes over GF(4), since the 
additive group of GF(4) is naturally matched to the QPSK constellation generating group, and 
Raptor codes provide the necessary rate flexibility to adapt the system to the variations of the 
source entropy rate, which may vary significantly depending on the symbol plane, the subband, 
and the image to be encoded. 

The linear mapping from source to channel symbols allows to avoid the use of a conventional 
entropy coding stage, as in conventional baseline systems, and this is expected to mitigate the 
catastrophic error propagation which affects conventional schemes in the presence of channel 
decoding residual errors. The proposed JSCC scheme is able to achieve pure image compres- 
sion performance almost identical to the state-of-the art. While the proposed system for finite 
block length and transmission on the AWGN channel yields lightly worse bandwidth efficiency 
performance than the highly optimized baseline system used by JPL in deep-space missions, the 
results of Sec. [VI] show that, as expected, the new scheme has much improved robustness against 
mismatched channel SNR conditions. While the baseline system requires the retransmission of 
a whole segment in the presence of even one frame with residual post-decoding errors, the new 
scheme yields perceptual good image reconstruction quality for SNR mismatch up to 1 dB below 
its nominal value, without any retransmission. 

Appendix 

A. Isomorphism 

In Sec. [V] an isomorphism between JSCC and channel coding over the two-block composite 
channel has been shown under MAP decoding. In this section, the isomorphism is established 
under BP in the sense that at every iteration of the decoder, the set of messages generated by 
the message-passing BP decoder for the source-channel coding scheme can be mapped into the 
corresponding set of messages generated by the message-passing BP decoder for the associated 
channel coding scheme by a probability-preserving mapping. 
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To this end, BP equations for a systematic Raptor code over GF(g) with K input symbols 
and K + n output symbols are given similar to |fl9ll . The Tanner graph corresponding to the 
systematic Raptor code is given in Fig. ©. Let G = [I; H] denote the encoding matrix of the 
linear code formed by the Tanner graph of the systematic Raptor code. Then the codeword vector 
d = [u c] = uH has length K + n, where u is the message vector. 

Let us consider the I th iteration of the BP decoder. BP messages can be represented in both 
probability and LLR domain as discussed earlier. In order to prove isomorphism, next we work 
in the probability domain where the messages are probability mass function (pmf) vectors of 
size q with the following notation: 

• ^m v o and ^m v are the messages passed from the v th input node to the o th output node 
and from the o th output node to the v th input node, respectively, of the LT-decoder; 

• ^m v c and ^m c v are the messages passed from the v th variable node to the c th check node 
and from the c th check node to the v th variable node, respectively, of the LDPC decoder; 

• ^ldpc i s me message generated from the v th LDPC variable node and passed to the corre- 
sponding input node of the LT-decoder; 

• S[^' v is the message generated from the v th LT input node and passed to the corresponding 
variable node of the LDPC decoder; and 

• t Q is the input message to the BP decoder at the o th output node. This can be either the 
a-priori source probability or the posterior symbol-by-symbol probability given the channel 
outputs. 

In case of a joint source-channel coding scheme the a-priori information on the first K 
symbols are obtained from source statistics. Assume the source symbols, w v 's are i.i.d. 
selected from GF(g) with Pr{u v = g} = Pu(g) f° r all 9 £ GF(q) and 1 < v < K. Let 
P[/ = [Pu(0), • • • , Pu(q — l)] be a vector of size q representing the pmf vector for the source 
distribution. Since the linear encoder is systematic 



t = Pi,. (25) 

On the other hand, for the two-block composite channel scheme, the t Q for the first K 
symbols are calculated using the discrete channel transition probability of the GF(g) additive 
noise channel. In this model, the transmitted vector x is equal to the codeword d and the 
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received vector is y = x — z where the operation is in GF(q). The noise symbols z 's are 
i.i.d. selected from GF(g) with Pr{z Q = g} = Pu{g) f° r a U 9 G GF(q) and 1 < o < K 
where the noise distribution is the same as the source distribution Pj/. Then, 



t a ,k = Pr{y \x = k} = Pu(y - k) = Pu(d Q -z -k). (26) 

For both the JSCC scheme and the composite block channel scheme, the codeword symbols 
d Q for K + 1 < o < K + n arc first mapped to points in the constellation signal set X by 
/J,(c ). Hence y Q = fi(d ) + z Q is received at o th output node where z Q E CJ\f(0, N ). Let 
the pdf of the complex circularly symmetric AWGN noise is denoted by f z . Then 

t ,k = Pr{y \/i(d ) = fi(k)} 

f z (\Hd ) + z -^k)\\) m , 
Y,k>f*{Md ) + z -n{k>)\\) 
Note that in AWGN initial channel message depends only on the distance between the 

observed vector and the hypotheses vector, hence we first focus on the calculation of the 

distance term, ||/i(g? ) + z — [i(k)\\. 

\\/i{d ) + z - fi{k)\\ = \\w do {v{0)) + z o -v{k)\\ 

= I IMO) + w_ do (z ) - w_ do (n(k)) 1 1 
= | |/x(0) + W- do (z Q ) - ft (k - d ) \\. 



(28) 



Substituting back into (1271) . we obtain 

fz (11/^(0) + W- do (z ) - jJ, (k - d ) 



^o,k 



Zk'fziwm+v-do^-vik'-djwy 

Next we investigate the relationship between input vectors t of two different scenarios which 
has been already discussed in Sec. |V] to be isomorphic to each other. For convenience, we 
name the two-block composite channel as Scheme A and JSCC as Scheme B. 
Scheme B: This case corresponds to the joint source-channel coding problem where the 
source vector u whose pmf is given by is encoded by G and d = uG and the non- 
systematic part of the codeword is transmitted through the AWGN channel where additive 
noise vector is z. Then the st for scheme B is given directly using (1251) and (T28l) : 

j Pu(k), if 1 < o < K 

Bt °' k = I ^ / 1^^:- d " (2 ; ) T ( ^° ) ] l \ii^ ot^rwise. ^ 
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Scheme A: This case corresponds to the composite-channel coding problem where all zero 
codeword (d = 0) is transmitted through the composite channel. For the additive GF(g) 
channel we pick the noise pmf as Py. The noise realization for this channel is taken as 
u (the same as the source vector u used in Scheme B) and for the AWGN part the noise 
vector components are w_d ( z o) for the o th output for K + l<o<K + n. Notice that 
pdf of the complex circularly symmetric AWGN noise in Scheme B and Scheme A are the 
same since, the transformation is either a rotation or a reflection. 
Then the A t 0>k for scheme A is given as follows directly using (T26l) and (|28l) . 

j Pu{u + k), if 1 < o < K 

At °' k = | .fz(MQ)+^-M-Kk)\\) otherwise (30) 
I Efc'/*(IMo)+™- d o(*o)-Mfc')ll)' omerwise - 

Next, we want to relate A t and #t in order to derive the relationship between BP messages 
in schemes A and B in Theorem [2l To this end, we define a shift operation + on pmf vectors 
as follows lfT9ll : Let g be an element of GF(g) and m = [m , m 1; . . . , m g _i] be a pmf vector 
of size q where the indices i = 0, . . . , q — 1 of each vector component are also interpreted 
as elements of GF(g). Index i denotes the i th element of GF(g) given some enumeration of 
the field elements where indices and 1 are reserved for the zero and one elements of the 
field, respectively. Then m +9 = [m g ,m g+1 , . . . ,m q ^ 1+g ] where summation is in the field. 
Then comparing (T29l) and (1301) . it is immediate to see that 

A t~ d ° = B t, 1 < o < K + n. (31) 

In the following theorem, we will prove that for any two schemes where t's are related by (1311) . 
the BP messages are related by (1331) and (1341) . 

Theorem 2: Assume the input probability message vectors of two different schemes, A and B 
are related as follows: 

A t Q d ° = B t Q , (32) 

where d Q is the value of the transmitted codeword d = uG for scheme B at the o th location. 
Then at any round I, the relationship between the messages passed in schemes A and B are as 
follows: 

( °b ov = (0 a o 7 v (33) 
« +1 )b vo = (34) 
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(where a is used to denote messages for Scheme -A and b is used for Scheme -B) and u v is the 
value of the v th variable node at scheme B. 

Before proving the theorem, next we introduce some useful notation and functions that will 
be helpful to manipulate probability domain BP equations. The BP equations in the probability 
domain can be written in a compact way using vector shift operations and Discrete Fourier 
Transform (DFT) as done in |fT9l . One of these shift operations is x of |[T9l which is similar to 
the previously defined + where summation is changed with multiplication in the field, i.e. m xg = 
[m ,m 9 ,m 29 , . . . , m( g _i) fl ]. The following properties from lfl9l are useful for our derivations: 

l^)^" 1 =m, ifg^O 



(m+*)~ 9 = 


m, and 


(m + ') X9 = 


(m^) + 


(m x f) +J = 





In the following we use e T d to denote the scalar product of two vectors while e • d denotes 
componentwise multiplication which results in a vector. Similarly Y[ denotes componentwise 
multiplications of multiple vectors. Additive vector representation U of g is denoted by the r- 
dimensional vector g. r-dimensional DFT and IDFT operations |fT9ll , G6l for vectors of size 
q = p r is described using the DFT pair f , DFT(f ) = d: 

d 9 = E A ej(27r/P) " T£ > 0<g<q-l, 

h£{0,...,p-iy 

fh = - Yl d g e-^^, 0<h<q-l. 
q ffe{o,..., P -i}<" 

We also define a new function T(-) and prove its properties which will be useful later in the 
proof of the theorem. Let T(-) be a function from GF(g) to complex vectors of size q where the 
h th component of the resultant vector is given as follows: 

[r(g)] h = e-^,0<h<q-l. (35) 



8 Note that finite fields exist for values of q equal to p r where p is a prime number and r is a positive integer. Each element of 
GF(p r ) can be represented as an r-dimensional vector over {0, . . . , p — l} r . The sum of two GF(p r ) elements corresponds to the 
sum of the vectors, evaluated as the modulo-p sum of vector components. This is called the additive vector-space representation. 
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Then it is easy to show the following properties of T(-) function: 



DFT (e +9 ) = DFT (e) ■ T(g) 
[IDFT(d)] +9 = IDFT (d ■ T(g)) 
T(g)T(h) = T(g + h) 



(36) 
(37) 
(38) 



We let g ov = g vo E GF(q) denote the value of the edge between the nodes o — v and define 
N(v) as the set of output o' nodes adjacent to the node v. Note that we use the same N(v) 
notation to denote the set of check nodes adjacent to v for LDPC part equations. It will be 
self-evident from the equations which set is considered. Similar neighbor notation N(-) is also 
used for other node types. Using the notation above, the updating rules for the LT and the LDPC 
decoders for the I th iteration are given as follows: 



OV 



IDFT (][,.. DFT f «m* v ° 



L v'eN(o):vVv 



DFTfe^" 



in, 



(i/g)i, 
u 



"ldpc ll 'e^^,A.„/j„ Hin'x 



if / = 0; 
otherwise, 



(39) 



(40) 



L 

where the operator U normalizes the vector, in other words U [m] = m/ (l T m). 

(l/g)l, if/ = 0; 



(0 



U 



f(0,v 



ric'eN(v):cVc ' 1 ^ m c'v 



otherwise, 



(0 m x-9c 
iii. cv 



IDFT 1 DFT ( ®m* a /< 

\ ll T '6N(c):v'/v 



The messages and S^ c passed from the LT to the LDPC decoder and from the LDPC to 



the LT-decoder respectively are defined by: 



:(0,v 



U 



n 



oeN(v) 



m, 



f(0,v 
'ldpc 



U 



n 



ceN(v) 



m, 



Proof: The proof is based on induction on (/). The relationship between the messages 
corresponding to different schemes for the th round, is verified first. Then round (/ + 1) will 
be proven assuming the hypotheses for Z th round. 
Round 0: 
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Using (I39H41I) it is immediate to see that 

Hence (|33l ) and (|34l ) are verified for / = 0. Now let's assume the theorem is true for I th round, 
and prove it for (7 + l) th round using (1331) and (|34|) . 
Round / + 1: 

= IDFT (n v , eN(o)iv , A DFT ( <,+1, b.-f ^ ' DFT (b^""- 1 
= mFT (n v , 6N(0):v¥v DFT (( <,+1 'VoT' ; '') ■ DFT ((A-*)"'-"-' 
<J 'DFT f n „ DFT f ( """^ • DFT (( ,^Y^ 



Lv'eN(o):vYv 



'DFT (n v , 6N(0):v¥v DFT ('-"a--) . DFT ( A *<->-) . V M - U,, m ^ 

idft (n. EN(0):v¥> dft (<«> a ;,:-) . dft . r L j: 



v'€N(o):vYv 



( = } IDFT 



^ IDFT [ TT DFT ( (l+1) a*>?°) " DFT (^t^" 1 ) -1 

(A) f(* + l) a x- g< ^ + " vi? ™ 



+«v9vo 



' a ov 



L 'ov "ov 

where (a) is due to Z th round assumption (1341) ; (b), (h) are due to (1351) . (1351) . Properties of the 
function, namely (136H38I) are used to derive steps (/), (c) and (d). Step (g) is simply the 
corresponding BP equation (l39l) for Scheme A. Lastly (e) is due to the check constraint at the 
o th output node. 

Assume the relationship for LDPC part messages, namely ^b cv ,^ a cv / z+1 ^ b vc /' +1 ) a vc similar 
to (|33~T) . (|34|) is already given. Then (|42|) can be directly verified. For space concerns the 
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assumption on the LDPC messages is not proven since it can be actually easily done using 
a similar proof to the current one. 



*(J+1),V _ ( *(I+l),v 
S°ldpc - U°ldpc 



(42) 



Then using (1401) . we write: 

( ' +2) b vo = U 



je('+i).v TT 



o'SN(v):oVo 



(i+l)b , 



A (I+l),v 
A°ldpc 



n o¥o 



(1+2) -u v 



(43) 



(I4TT) and (33) completes the proof. ■ 
Due to (13TI) . according to Theorem ©, the BP messages of Scheme A and Scheme B can 
be obtained from each other using the operation + . Hence it can be easily seen that the error 
probability of Scheme B with u and z is equal to the error probability of Scheme A with u 
and W-d (z ). Since AWGN is isomorphic, the average error probability of Scheme B (JSCC) 
is equal to the average error probability of the Scheme A (composite two-block channel). 



B. Stability Condition 

In this section we extend the stability condition of Etesami et al. (6l to LT codes over GF(4) 
and the two-blocks composite channel of Definition [2] with parameters H and C . Let F be the 
4x4 DFT with elements [F] m/ = e -i*(m-i)(<-i)/2 f or m> £ e {i ; 2, 3, 4}. The BP messages for 
g-ary codes can be either represented as probability vectors (of length q) or as LLR vectors of 
length q — 1. Let LLR : m >-)■ L denote the mapping of the probability representation into the 
LLR representation from the probability domain. We define the mapping $ : M 3 — > [0, l] 4 given 
by 

$(L) = F LLFrVD. 



Under the Gaussian approximation L ~ Af(vl, £„) of lfT9ll , already used in Sec. IV-A[ we have 

E[$(L)] = [1, V(v), V(v), *(v)}, (44) 

with 

a n - p~ Li a- p~ L2 - p~ L:i i 

(45) 



1 - e~ Ll + e~ L2 

1 -I- P~ L 1 -I- P~ L 2 



-La 
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Since the right hand side of (1441) depends only on a scalar value, when we consider the expectation 
of LLR values under Gaussian approximation, it will suffice to work with a "scalar" version of 
the expectation operator, denoted by E s . For example, E s [L] = v is the short-hand notation for 
E[L] = vl. With this notation, we have E s [$(L)] = = * (E s [L]). 

The BP message updating equations in the LLR domain, assuming an output node o with 
neighborhood N(o) of size |N(o)| = i and an input node v with neighborhood N(v) of size 
|N(v)| = j, are given by 



$ (LS? y ) 
L (i+i) 



i-1 



n*«'.»o) 



i=i 

j'-i 



$ft r 



j=i 

where t Q denotes the LLR of the channel output for node o, and / denotes the BP iteration. 



Following J6l, we are interested in the evolution of the quantity E s 
borhood of 0. We can write 



(0 



in a right neigh- 



E * W?v] =5> E * 1 |N(o)l =i] 

i 

= Yl^' 1 (* ( E * Mil |N(o)i = t])) 

i 

= X)^- 1 (E s [$ (L« ||N(o)|=i)]) 

i 

= E^ _1 { [7* (^(1 " #)) + (1 " 7)* (^(C))] [E. [* (Lg, )]] 



i-1 



[7* (J- 1 (l-i/)) + (l-7)^(^ 1 (C))] 



5>*(tf-l)E. [L^>]) 



i-1 



where in (a) we used the two-block composite channel property. For successful start of the 



(0 



must be strictly increasing 



decoding under the Gaussian approximation, the quantity E s 
from one iteration to the other in a sufficiently small right neighborhood of zero. A necessary 
condition is that 



v 



V 



i-r 



(46) 



in a sufficiently small right neighborhood of v = 0. By taking derivative of both sides of (1461) 
with respect to v at and using \P(0) = and ^'(0) ^ (see at the end of this section), after 



October 3, 2012 



DRAFT 



34 



some algebra we arrive at the stability condition: 

^2 > T (7, ri dpc , H, C) L 



7/ Hdpc 



(47) 



2 ( 7 * (j-i(i _ #)) + (1 _ 7 )* (J-i(C))) ' 
Calculation of ^'(0). We rewrite *f?(v) in (l45l) using the zero mean Gaussian random vector 

£ ~ J\f(0, E„) as 

= E 



1 - e - £l "" + e~ C2 ~ v 



-Ci—v 



(3 — C2—V _[_ g — £3— D 



Using a Taylor expansion in a neighborhood of i> = 0, we obtain 



where 77 = ni + n 2 + tt, 3 and 
h, 



00 00 00 

ni=0 n2=0 ri3=0 



ni!n 2 !r23! 



(48) 



l-e-'l-"+e-'2-"- 



l+e _l i -,, 4-e _J a— u +e -, 3- 



(49) 



ru,n 2 ,n 3 V", dl^dl^dlf 

h=h=h=0 

Let L be a zero mean Gaussian random vector of size 77 with covariance matrix /C = [/Qj], and 
let S r/ denote the collection of all unordered sequences of all unordered integer pairs, with each 
sequence containing each integer from 1 to 77 exactly once. Then, the well-known formula for 
the higher moments of real Gaussian random variable yields, 



E 



t=i 



(50) 



0, for 77 odd 

yV. . . /Q 7 , for 77 even. 

Specializing the components of L to be L m = C\ for 1 < m < n\, L m = £ 2 for ni + 1 < m < 
rii + n 2 and, Z m = £ 3 for n\ + n 2 + 1 < m < 77, using the form of E„ as given in Sec. |Vj and 
using (|50l . it is not difficult to see that E [C^C^C^] oc (H^W 2 for all even values of 77, and 
zero for all odd values of 77. Then, we have 



E 
E 



is. i„u/2 



ni\n 2 \n 3 



dli 



721 ,722,723 : Tj is even 



"i."3."3M .,n/2 1 H„. 77/2-1 1 c 7 .n 
ni!n 2 !n 3 ! 



For t> = 0, the above summations includes the triplets (ni, n 2 , ti 3 ) with sum 77 = ni+n 2 +n 3 = 2. 
These are (200), (020), (002), (110), (011), and (101). For any (n u n 2 ,n 3 ) in this set, we have 
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E[£™ 1 £™ 2 £ 713 1 

niirjin \ = V ' smce tne diagonal elements of E„ are equal to 2v and the off-diagonal elements 
are equal to v. Then, we obtain 

tt'(O) = ^V,n 2 ,n 3 (0) 

= h AO (0) + ^0,2,0(0) + /io,o, 2 (0) + ^1,1,0(0) + h oxl (0) + h lfitl (0) 
= -60/16, 

where, using d49l) , we have /i 2 ,o,o(0) = ^0,0,2(0) = -2, = 0, /ii )0| i = 1/8, foo,i,i(0) = 

-1/16 and /i ,2,o(0) = 3/16. 
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